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The International Comparative Study on the Organization and Performance of Research 
Units (ICSOPRU) is conducted under the assumption that it is not only an academically 
relevant study, but also one from which practical outcomes could be derived. It has always 
been assumed that ICSOPRU results could provide science policy decision-makers with 
specific instruments for action. This paper' is a preliminary reflection on this assumption, based 
on results from Brazil. 

1. Brazil's participation in ICSOPRU 

Brazil's participation in the ICSOPRU project began with an agreement between 
UNESCO and the Financiadora de Estudos e Projetos - FINEP - a public funding agency 
concerned with the development of scientific and technological activities. The execution was 
entrusted to the Instituto Univesitario de Pesquisas do Rio de Janeiro (IUPERJ) a private, non- 
profit institute devoted to research and graduate teaching and sociology and political science. 

According to the terms of the agreement between FINEP and IUPERJ, Brazil's 
participation in the ICSOPRU project had as general objectives, "a) to identify a sample of 
research units within institutions devoted to scientific and technological research, and to 
interview its members by means of a set of questionnaires; b) to evaluate the importance of 
organizational, environmental and similar factors that may affect the performance of research 
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units in the country" In the long run the project aims were said to be "a) to provide decision- 
makers involved in the formulation and implementation of national science and technology 
policies with information concerning the performance of research groups obtained by means of 
an internationally tested methodology; b) to suggest strategies to improve the administration of 
scientific research in the country; c) to organize data sets and to develop indices of 
effectiveness, productivity and organizational climate within research units". 

The project started in Brazil in early 1982, and ended in December 1984. The first year 
was devoted to the adaptation of UNESCO materials to Brazil, development of national 
addenda identification of a national sample and training of interviewers. The second year was 
devoted to fieldwork, organization of computerized data sets and writing of preliminary 
reports. In 1984 part of the original research team continued to work with data analysis, data 
cleaning and international compatibilization of the Brazilian results. Several papers circulated 
that year, and a final report in Portuguese is being finished and delivered to FINEP in mid 
1985. In January 1985 IUPERJ organized with FINEP an International Seminar on 
Development, and Scientific and Technological Effectiveness with participation of ICSOPRU 
and non-ICSOPRU researchers and science policy-makers from Brazil, Argentina Colombia, 
Mexico, the United States Canada Finland, Sweden. The Netherlands and Ukraine. 
Presentations of ICSOPRU' s preliminary results were made in several Brazilian governmental 
and university institutions, including the National Research Council, FINEP and the Catholic 
and Federal Universities in Rio de Janeiro. 

2. Policy-making and social research 

Social scientists often declare their studies to be geared to effective utilization in 
policymaking, while policy-makers support research on the assumption that results could be 
useful for their purposes. However, effective utilization of research results by policy-makers is 
rare. Several reasons have been suggested to explain this state of affairs. 

A - Social science research is often couched in terms not amenable to the decision- 
maker's intervention. In fact, decision-makers have only a limited number of variables they are 
able to manipulate. If a study concludes that variables such as "cultural heritage", or 
"dependency" are responsible for some undesirable facts, policy-makers can agree, but can 
usually do nothing about them The reason social scientists deal with this type of variables is 



that they are often more interesting from an interpretive point of view; and also because they 
usually lack theories that utilize less far—reaching variables. 

B - Social sciences' statements are usually too imprecise to be amenable to immediate 
utilization, even if variables are well defined and empirically tested. Quantitative results are 
typically presented in terms of low correlation coefficients, and there are always problems with 
assumptions about scalability, universe and sample distribution curves, multicolinearity, and so 
forth. Quantification does not mean precision. 

C - If social scientists are often unable or unwilling to provide decision-makers with 
readily useable results, decision-makers are often not prone to take social research results into 
consideration. Priorities may be different, political constraints too strong; or decision-makers 
may lack the proper' personnel and organization. 

D - Social scientists can also provide answers policy makers are not willing to listen, 
and do not answer what they want to hear. For instance, policy-makers may want to have clear 
cost—benefit assessments of investments, or decision criteria about alternative uses of public 
fun'-Js The typical researcher will answer that there are many different dimensions of 
effectiveness and that long-range scientific research is not amenable to cost-benefit analysis. 
He will also say that there are no clear— cut, technical criteria to decide, for instance, whether to 
invest in nuclear or in biological research since it is a matter of political choice He may suggest 
an improvement of the peer-review system or an enlargement of the agency's constituency. 
Policy-makers can perceive these suggestions as attempts to limit their authority. 

E - Timing is also different. The first negotiations for the ICSOPRU project in Brazil 
started in 1981 or before, and the main results started to appear in 1984-5. During these years, 
there were several changes in FINEP's administration, and a profound transformation in the 
Brazilian government. There is little memory in FINEP today about the original purposes and 
potential use of ICSOPRU. Lack of continuity due to changes is not just an effect of different 
people, nor necessarily a consequence of large reformulations of previous concerns and policy 
orientations. It is mostly a consequence from the lack of well-established administrative 
organizations and procedures that can carry on with their tasks while changes are made at 
higher levels. However, large institutional changes are in fact occurring at this moment. The 
creation of a Ministry of Science and Technology by the Brazilian new government was 



supposed to strengthen its commitment with this field; in fact, however, the new Ministry is 
still being organized, and it is not clear how it will proceed in terms support for science and 
technology. In contrast, a research project can continue for several years, relatively unhindered 
by this changes. 

Given all these difficulties, one can wonder why the notion that social sciences can 
provide instruments for active policy making is so commonly sustained. One reason is the 
sincere concern of social scientists with effective use of their findings, and of policy-makers 
with improving the quality of their decisions. It also happens that practical justifications can 
increase the probability of obtaining support for social research, and scientific research can 
legitimize policy decisions taken on quite different grounds. Either way, there is a clear process 
of mutual attraction that produces pragmatically oriented research projects, without necessarily 
leading to applied results 

It is clear from the previous discussion that the eventual utilization of results of a given 
research project depends only partially cm the project's contents. Even the best-applied study 
would not be utilized by an unstable, unconcerned or incompetent agency; even poorly 
conceived studies could yield significant information for an active, competent and properly 
motivated policy institution. The following discussion will be limited to one side of this 
problem, namely what kind of results from ICSOPRU could be possibly be applied, and how. 
In other words, 1 1. will deal with the researcher's side, not with the decision-maker's one 

The first two tasks given to the ICSOPRU project in Brazil described the work IUPERJ 
had to do according to the UNESCO-FINEP agreement. A sample of research units was indeed 
identified, the UNESCO and national questionnaires were applied, and analyses were made 
evaluating "the importance of organizational, environmental and similar factors that may affect 
the performance of research units in the country". Since "performance", or effectiveness, is the 
main dependent variable, one should start by discussing it. 

3 The measurement of performance 

Performance" or research effectiveness appeared to be a multidimensional 
phenomenon. This means that different research units work for different goats and evaluate 
themselves and each other, according to different criteria that cannot be reduced to a single 



additive index. This fact was observed in all countries where the project was carried on. The 
main dimensions of effectiveness are: 

A - academic effectiveness, expressed in international publications and concern with 
purely scientific results. In Brazil this is typical of university research units in the fields of 
biological and exact sciences. 

B - applied effectiveness, expressed in national publications and practical/social 
significance of research. In Brazil, this was most typical of agricultural and medical research 

C - technological effectiveness, expressed by the production of experimental devices 
and patents. It is typical of technological research units in universities and institutes. 

D - "internal" effectiveness, expressed by the production of non-published internal 
reports and compliance with the parent organization's norms and procedures. It is most typical 
of research units belonging to productive companies 

ICSOPRU's indicators for these dimensions have several limitations. Some of these 
indicators are "objective", in the sense that they refer to numbers of items produced in the 
different dimensions. This type of information suffers from all criticisms that have been 
addressed to counting articles, etc., to measure effectiveness. These problems become more 
acute as one moves from international journals that are supposedly more selective, to national 
periodicals, where the frontier between truly "scientific" and non-scientific journals becomes 
fuzzier. To count "experimental devices" and "internal reports" is still more fuzzy in terms of 
what the answers provided actually contain. There are some ways of improving the quality of 
these measurements: to work with citations and to weight the journals according to their 
academic standing, for instance, or to evaluate the worthiness of technological products. These 
improvements are however impractical for ICSOPRU's scale, and were not attempted. 

Other indicators are "subjective", in the sense that they give the units ratings along 
several items according their own leaders and members In the beginning, attempts were made 
by ICSOPRU teams check these internal evaluations with ratings done by external judges. 
However, problems of sampling external evaluator's are almost insurmountable (they may not 
know enough about the units if selected randomly, or be too biased if nominated by the units 
themselves). In those countries where external evaluation was performed, it was found that 



they did not differ from internal evaluations, and were usually less severe. Because of this, the 
decision was made not to proceed with the external evaluations. 

In order to test for the quality of these indicator's in Brazil, the units were distributed 
according to whether they ranked high or low in several dimensions of effectiveness, and these 
results were compared, for units in universities, with ratings their institutions (departments or 
programs) received from CAPES, an agency from the Ministry of Education which provides 
support for graduate studies. The results can be seen in table 1. 



Table 


1 


- Average rating by CAPES(*) for Research Units in universitie, by 
different effectiveness dimensions (N = 150) 

efectiveness probability(**) 


high low 



articles published in the col 


intry 


1.73 


1.95 


articles published abroad 




1.60 


2.04 


experimental devices 




1.50 


2.15 


training effectiveness 




1.90 


1.77 


national reputation 




1.82 


1.92 


social relevance 




2.02 


1.81 


follow up 




2.03 


1.78 



0.28 
0.02 
0.00 
0.27 
0.60 
0.31 
0.23 

(*) CAPES ratings are given to graduate programs by peer review committees which 
receive full documentation about the programs' activities, publications, dissertations 
produced, etc. They vary from A (1) to D or E (4). For this test, we attributed to the 
units the ratings received by the programs to which they are associated. 

(**) Provided by analisys of variance, it expresses the chances that between-group 
differences are not different from within-group differences. 

What this table shows is that CAPES's ratings fit well with two indicators of 
effectiveness, production of experimental devices and articles in international journals. They fit 
also, although more weakly, with production of articles in national journals. This means that, in 
spite of their limitations, these indicators can be taken as reasonable references to the unit's 
performance. For the remaining indicators (except reputation), however, the relationships are 
non-existent or negative, that is, the groups that rate themselves higher get lower CAPES 
marks! There are two alternative ways of interpreting these results. The first is that self-ratings 
are self-serving, and cannot be taken as measures of effectiveness. The second is that CAPES' 
evaluations are biased towards measurable products and do not account for more qualitative 
realities captured by ICSOPRU questionnaires. In any case, this test casts a shadow of doubt on 
the true meaning of self —rated effectiveness indicators. 

We can conclude that ICSOPRU data measure reasonably well in aggregate terms, what 
science indicators usually grasp, that is, academic performance. The quality of these indicators 
becomes fuzzier as we move to other dimensions, and it is practically impossible to say what 
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"internal effectiveness" means although we can suspect that, in this case it may have little to do 
with objectively effective science and technology. 

4 Environmental and other factors affecting performance. 

Two environmental factors have a clear impact on effectiveness of research units, 
namely the type of institution and the geographic region where they are located. These two 
variables, plus the different fields of knowledge were utilized as predictors. In an analysis 
having as dependent variables the production of articles in the national and international 
literature (the analysis was performed using the program "Search" from the OSIRIS IV 
statistical system; results are presented in tables 2 and 3 in tree form). 



Table 2: Influence of field of knowledge, institution and regional location on the production of articles in national publications. 

(total of explained variance, 14.3%) 



Group 1: total 

Mean: 9.6 

N=279 



J_ 



Group 2: Exact, biological sciences, and technology 

Mean: 7.1 

N=194 



r 



JL 



Group 4: 

Companies 

Mean: 2.2 

N=32 



-I 



Group 5 

Institutes and universities 

Mean: 8.1 

N=162 



Group 3 - Medical and agrarian sciences 
mean: 15.8 
N=85 



JL 



Group 6: Rio and Northeast 
Mean: 6.1 
N=29 



Group 7: S ao Paulo and Center South 

Mean: 19.9 

N=56 



Table 3: influence of field of knowledge, institutions and regional locatgion on publication of articles in the international literature 

(explained variance: 11%) 



Group 2: institutes and companies 
mean: 1 .6 
N+113 



Group 1 

Total 

Mean: 3.5 

N:279 



J- 



Group 3: universities 
Mean: 4.9 
N:166 



Group 4: Rio, Northeast and Center South 
mean: 3.4 
N=116 



n 



Group 5: S ao Paulo 

mean: 8.2 

N=50 



Institutional factors seem to have mostly a negative impact on production of published 
materials. Units in companies produce least, while units in universities may produce more. 
International publications are limited to university research units. High productivity is not 
explained by institutional factors, but by "geography": a condition for high publication is to be 
located in Sao Paulo or at least, for articles in national journals, in the Center-South region of 
the country. Field of knowledge influences mostly what kind of product one has national 
publications for agriculture and medicine, international publications for biology and physics, 
few publications for technology. What is behind "geography" is, of course, economic and 
social development. 

Policy implications of this finding are many. In spite of all difficulties, universities 
seem to be best location for research work, either of applied or of more academic orientation. 
The advantages of university research are first, that they can provide a good environment for 
"entrepreneurial" activities of unit leaders, which are more restricted in their initiatives in non 
university institutions; second, they have the presence of graduate students; third, the reward 
system for university research can be more closely adjusted to scientific achievements than the 
reward systems in non university environments. This means that the tendency of the Brazilian 
government in the last several years to invest heavily in non-university research might have 
been misdirected. However, to be in a university environment is not enough: it has to be a 
"good" university, with characteristics that are usually found only in the more developed 
regions in the country. It is interesting, in this regard, to notice that this is not a simple matter 
of economic resources, since Rio de Janeiro appears in this analysis closer to the Brazilian 
Northeast than, for instance, Minas Gerais. What are the characteristics of "good" institutions? 
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4. Organizational and administrative factors 

Administrative factors seem to play a relatively unimportant role in the performance of 
research units. Tables 4 to 6 summarize the main findings. Factor analyses were performed for 
questions J, K and Q of the ICSOPRU core questionnaires. Question J deals with leadership 
patterns of involvement, question K with working climate and question O with organization 
and planning of research work Then, factor scores were produced and an analysis of variance 
was utilized to calculate aver-age scores for units high and low in different effectiveness 
measures (the same as in table one) The tables gives only the values found for the highly 
effective groups in each dimensions, and asterisks indicate if their differences with low 
performing groups are significant. 

Factor analysis indicates that leaders get involved in the unit's work along three 
dimensions: project's conception and design, research work and research administration. Of the 
three, only involvement in project's conception and design relates with effectiveness in its 
many dimensions The more significant relations, however', are with subjective effectiveness, 
not with quantitative measures. Leader's involvement with actual research work bear no 
relationship with effectiveness, and involvement with research administration seems to have 
only minor impact. Patents production appears very rarely in the Brazilian study, and seems to 
be the product of odd, individual initiatives this may explain the negative relation it has with 
the leader's involvement with the first factor (table 4). 

Table 4: involvement of leaders with different dimensions of research 

leader's participation in 

project's 
conception and research research 

design work administration 

Dimensions of effectiveness: 

articles published in the country 

articles published abroad .30* 

experimental devices 

patents produced -.53** 

training effectiveness 

Innovativeness 

national reputation 

General contribution to C&T 

social relevance 

follow up 



44** .21* 

55** 

38** 

56** .30* 

44** 



Factor 


scores 


are 


sta 


ndardized. 


The 


table 


shows 


only 


va 


ues 


where 


there 


are 


*: p< 


10 


























** p < 


.01 



























The units' working climate includes four dimensions: a general pattern of more or less 
conflicts; the existence of technical cooperation among members external (administrative, 
environmental) difficulties; and levels of internal participation (meetings, information sharing) 
Here again, only the second factor has systematic relations with effectiveness; but there are no 
relations with indicators of academic performance, while the relation with technological 
effectiveness (experimental devices) is negative In short, units where people are united by 
technical cooperation and a feeling of innovation perceive themselves as effective, which does 
not mean they are actually so More psychological and environmental conditions (feeling of 
participation, conflicts, external difficulties) seem to he irrelevant (table 5). 



Table 5: Working climate (question K) in highly performing units (N = 201) 

working climate 

technical external 

conflictiveness cooperation difficulties participation 
Dimensions of effectiveness: 

articles published in the country 

articles published abroad 

experimental devices -.30* 

patents produced .45* 

training effectiveness .58** 

Innovativeness .51** 

national reputation .42** -.33* 

General contribution to C&T .40* .30* 

social relevance .60** 

follow up -.28* .69** 

Factor scores are standardized. The table shows only values where there are significant 
differences between high and low performing units. 
*: p<.10 
** P <.01 



Table 6 follows the same pattern. Four dimensions of organization and planning of 
research work were identified: quality of research work, quality of follow up and applications, 
high participation and unified budget. The most important is quality of research work, secondly 
the quality of applications and follow up (the relative importance of each factor is given by the 
percentage of total variance of items explained by its factor loadings). High participation tends 
to have mixed, but predominantly negative impact on effectiveness the same happening with 
unification of the units' budget an independent factor of minor (and negative) weight. 
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Table 6. Organization and planning characteristics of highly performing units 



Dimensions of effectiveness: 

articles published in the country 

articles published abroad 

experimental devices 

patents produced 

training effectiveness 

Innovativeness 

national reputation 

General contribution to C&T 

social relevance 

follow up 



(N = 238) 










charcteristics 




good 


good follow- 






research 


up, 


high 


unified 


work 


appications 


participation 
-.27* 


budget 


.32* 


.35* 










-.27* 


-.26* 






-.35* 




72** 








.65** 


42** 






.67** 






-.16* 


.50** 


47** 






.37** 


.69** 


.33* 


-.29* 


.31* 


.51* 


.26* 





Factor scores are standardized. The table shows only values where there are significant 
differences between high and low performing units. 
*: p<.10 
** P <.01 



In short, the analysis suggests strongly that effectiveness as a subjective feeling is 
related with perception of quality of the unit's scientific and technological work, and its leaders' 
involvement with research design and conception; that purely administrative or socio- 
psychological conditions are irrelevant; but most fundamentally, that actual effectiveness tends 
to have little relationship with all these variables The policy implication of this finding is clear: 
there is no point in investing money and effort in the improvement of administrative conditions 
or on the so-called "organizational development" of research units; they either have scientific 
competence - and it that case they would usually obtain the administrative resources they need 
- or they are hopeless. 

5. Individual characteristics 

One of ICSOPRU's basic assumptions is that scientific and technological research is a 
group not individual process. This is certainly an improvement over conventional empirical 
sociology, which tends to work with individuals as its main unit of analysis. Sociology of 
science and common knowledge, however, usually emphasizes the role of individuals in 
scientific and technological production as leaders, innovators, and researchers. 

A test of the individual vs. group hypotheses can be done through table 7. Regression 
analyses were made with 1,112 scientists and engineers interviewed in the project. They show 
that articles in the international literature are, among all items, the ones more strongly related 
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to individual achievement. They have the highest correlation with doctoral degrees; show little 
or no influence from the institution where the scientist works, from his role (whether he is a 
leader or not), or from his field of knowledge. 

Table 7: determinants of individual productivity (multiple regression analysis) 



position as unit leader 

male 

fully dedicated to research in the unit 

fully dedicated to teaching and research 

does research within and outside the unit 

has a doctoral degree 

unit belongs to a university 

unit belongs to a company 

technology 

exact and earth sciences 

medical research 

agricultural research 

has studied abroad 

present work related to the field of eduction 

Sao Paulo 

Rio de Janeiro 

Northeast 





Prod 


ucts 








articles in 


interna 


patents 


or 




international 


routine 


patents 






journals 


reports 


requested 


0.138 


0.088 




0.127 


0.061 


0.013 


-0.011 
0.006 




-0.090 


-0.024 


0.107 




-0.052 


-0.024 


0.002 


-0.002 
0.007 




-0.019 


0.008 


0.020 


0.012 


0.081 


0.158 


0.290 




-0.061 


-0.041 


0.064 


0.045 
-0.061 




-0.173 


-0.115 


0.134 




-0.360 
-0.036 


-0.043 


0.014 


-0.063 


0.039 


0.450 


0.003 
-0.011 




-0.078 


0.022 


0.117 




-0.031 


0.058 


0.179 


-0.035 




-0.005 
0.013 
-0.005 


-0.039 


0.010 


0.116 


0.048 


0.087 


0.090 


0.016 


0.033 


0.079 




0.006 


0.024 


0.078 


0.078 




0.007 


0.037 


0.002 


0.000 




-0.005 


0.015 



(1) 


y=l 


og(l+x) 


*** 


P 


<.001 


** 


P 


< 010 


* 


P 


<.100 



Articles in national journals and in Portuguese, however, follow a very different 
pattern. They also depend on a doctoral degree, but to a lesser extent. They tend to increase if 
the researcher is a unit leader, works in a research institute in the fields of medical and 
agricultural sciences, and is not dedicated full-time to research work in his unit. These articles 
are probably less academic in nature, and the journals that publish them are likely not to be 
strictly scientific or technical. 

Internal routine reports, at the other extreme, are strongly dependent on institutional 
variables. To produce them, one has to be a scientific leader, female (it is impossible to explore 
further the meaning of this finding here), to work in a company, not in a university, not to have 
a doctoral degree, and not work in the exact or earth sciences. 

Technical products have patterns of their own. Patents, which are extremely rare, occur 
outside the universities, and are made by people who have studied abroad and work both within 
and outside their main institutions. There is also some concentration of patents in the medical 
(presumably, pharmaceutical) area. The impression is that the few patents that exist are odd 
events that depend on the work of a few independent, free-lance individuals. Experimental 
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devices, however, have clearer determinants. Mostly, they depend on whether one works in the 
fields of technology and exact sciences, which usually means, in Brazil, that one is male. They 
also require a shift from some previous specialization into a new one. Given these conditions, 
this type of product seems to come naturally. 

In short, some scientific products depend on personal achievement along the academic 
path, others on one's institutional and disciplinary insertions; others, like patents, do not seem 
to have found a definite place or a source for incentive. These products are not interchangeable: 
those with a clear scientific and technological content are more clearly related with individual 
characteristics than those expressing routine or bureaucratic work Policy implications of these 
findings are self—evident, and in line with previous conclusions: the locus for scientific and 
technological achievement seems to he essentially the individual. If competent researchers are 
obtained, and if they can be provided with space to develop and adequate support good 
research is likely to be forthcoming. Then it will be important to ask about the role of 
organizational institutional and environmental supporting conditions. If not, all this is quite 
irrelevant. This conclusion seems to be trivial for anyone who is a professional researcher. 
However, it is not so self-evident for those in charge of science policy and policy 
administration in so many countries striving to build up their scientific potential. 

6. ICSOPRU and administrative support: information, strategies and indices 

The ICSOPRU project in Brazil can, therefore, point to some basic strategies to be 
followed by policy-makers, and to some dead ends in previous conceptions on how to go about 
this task. The ambition, however, was higher: the project was meant, as stated in the beginning, 
to provide decision-makers with specific tools for their day-to-day activities. 

The first of these tools were to be "information concerning the performance of research 
groups obtained by means of an internationally tested methodology." This tool cannot be 
provided for two reasons. First, the ICSOPRU project is based on clauses of confidentiality, 
and no information on specific units could be provided to policy-making authorities. Secondly, 
as we have seen, the measurements utilized in the project are useful only for cross-section 
analyses. They are useful to make broad comparisons and to test general hypothesis, but are not 
discriminating enough to provide precise evaluations of specific units. In fact, the ICSOPRU 
study in Brazil spotted a few areas that seem to be particularly problematic. For instance, units 
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of medical research in general and agricultural research in the universities show several 
indications of trouble, and units in large, state-owned companies are probably quite ineffective 
in comparison with their costs It is difficult, however, to go much beyond these general 
statements, and ICSOPRU is not a good instrument for continued monitoring of performance. 

The international validity of ICSOPRU's indices is also open to questions. It is still not 
clear, at least for this author, how useful is the international component of the project. Most of 
the analyses done so far with ICSOPRU data have been national; when comparisons are made, 
they have seldom gone beyond merely descriptive tables. Sometimes units of different 
countries are lumped together, but the "country variable" is seldom taken into consideration. A 
lot of effort has been placed in assuring item-per-item comparability among national studies. 
However, there are some basic uncertainties on how the definition of "research unit" was 
adopted in different countries, and on how the institutional variable was considered. The 
meaning of products such as "articles in national literature", "books published" and "patents" 
vary widely among countries. They depend on the historical linkage a country has with 
international research centers, the existence or not of local journals, the local concept of 
"book", the state of national book industry, the national patent legislation, and so forth. As the 
ICSOPRU project migrates from Western Europe to Eastern Europe, Latin America, Asia and 
Africa, difficulties with assuring item-by-item comparability will tend to increase, and its 
worthiness are bound to become more problematic. There is an obvious problem of 
diminishing returns in this process, which should be taken into account. 

The second tool would be "to suggest strategies to improve the administration of 
scientific research". Some strategic considerations have already been mentioned. They refer to 
policy contents, since there is nothing in the project that deals directly with the way policy- 
makers should organize them for implementing these or other strategies. However, if the 
conclusions about the importance of individual and leadership factors is warranted, a point 
could be made for the need for science policy agencies to rely as much as possible on peer 
review mechanisms instead of on purely organizational or administrative procedures of 
evaluation and research support, since peers are obviously better at identifying quality than 
civil servants. 

Finally, there is the suggestion that ICSOPRU could provide decision-makers with 
"data sets and indices of effectiveness, productivity and organizational climate within research 
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units". This can also be interpreted in terms of a system of permanent monitoring. Even if the 
confidentiality clause were raised for, future ICSOPRU-like studies, one could still wonder 
whether this kind of semi-permanent evaluation would not be too costly and a source of 
unnecessary nuisance and friction between scientists and policy-makers. General receptivity 
for the ICSOPRU project was good in Brazil, but it would probably deteriorate if it were to he 
repeated every two or three years More fundamentally, it is clear that monitoring of scientific 
performance can only be done through active participation of the scientific community, and 
requires specific evaluations of contents of the results obtained, something which is beyond the 
scope of cross-section studies such as this. 

7 Conclusion 

ICSOPRU is certainly a good instrument to provide an overview of the way scientific 
work is organized in different settings and countries. It requires fairly sophisticated analytical 
skills, and its potential for international generalizations lies probably more on comparisons of 
findings than on item-by-item tabulations. In spite of its original emphasis on group theory, it 
contains important individual-based information that should not be neglected. If properly 
analyzed, this project provides non-trivial insights. However, it is not adequate for furnishing 
decision-makers with ready-made information on a unit-by-unit basis or precise decision 
criteria on specific circumstances. It says nothing about how science policy agencies should get 
organized to fulfill their tasks. 

How decision-makers will use these results depend on their ability to understand the 
project's conclusions and their freedom to act. Given the general difficulties in the relations 
between policy-making and social research, it is probably an error to try to direct the analytical 
effort in a project such as this only to the decision-makers' benefit. If the research is well done 
and, more important, properly analyzed, results will appear', and help to form opinion. Then, if 
policy does not follow from accepted knowledge, it is for reasons that usually lie well beyond 
what researchers can do. 
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